Papers with downstream model
Data Collection for Dialogue System: A Startup Perspective (N18-3)
Copied to clipboard
| Challenge: | Developing dialogue systems such as Apple Siri and Google Now requires high quality training data but data collection with crowdsourcing is largely an open question. |
| Approach: | They propose to use crowdsourcing to collect data for a user intent classification task in a dialogue system. |
| Outcome: | The proposed method improves the quality of the collected data and the model performance on real user queries. |
A Multi-Level Optimization Framework for End-to-End Text Augmentation (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing methods for text augmentation perform data augmentation and downstream tasks separately. |
| Approach: | They propose a framework to perform text augmentation and the downstream task end-to-end. |
| Outcome: | The proposed framework performs text augmentation and the downstream task end-to-end on a text classification dataset. |
Joint Optimization of Tokenization and Downstream Model (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have reported that an appropriate tokenization depends on each downstream task. |
| Approach: | They propose a method to find an appropriate tokenization to a downstream task by optimizing a tokenizer and a model. |
| Outcome: | The proposed method improves on text classification and machine translation tasks. |
Retrofitting Contextualized Word Embeddings with Paraphrases (D19-1)
Copied to clipboard
| Challenge: | Contextualized word embeddings can be useful for downstream applications, but they can be over-sensitive to contexts. |
| Approach: | They propose a method to retrofit contextualized word embeddings with paraphrases to minimize the variance of word representations on paraphrased contexts. |
| Outcome: | The proposed method improves on sentence classification and inference tasks. |
Intrinsic Bias is Predicted by Pretraining Data and Correlates with Downstream Performance in Vision-Language Encoders (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent work has found that vision-language models trained under the Contrastive Language Image Pre-training framework contain intrinsic social biases, but how these biase relates to downstream performance has been unclear. |
| Approach: | They present the largest comprehensive analysis to-date of how upstream pre-training factors and downstream performance of CLIP models relate to their intrinsic biases. |
| Outcome: | The proposed model performance analysis shows that the choice of pre-training dataset is the most significant upstream predictor of bias, whereas architectural variations have minimal impact. |
Prompting ChatGPT in MNER: Enhanced Multimodal Named Entity Recognition with Auxiliary Refined Knowledge (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to enhance textual entity prediction neglect the need for external knowledge or encounter high redundancy in the retrieved knowledge. |
| Approach: | They propose a framework that leverages ChatGPT as an implicit knowledge base and heuristically generates auxiliary knowledge for more efficient entity prediction. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on two classic datasets and exhibits a stronger robustness and generalization capability. |
Can Humans Identify Domains? (2024.lrec-main)
Copied to clipboard
Maria Barrett, Max Müller-Eberstein, Elisa Bassignana, Amalie Brogaard Pauli, Mike Zhang, Rob van der Goot
| Challenge: | Textual domain is a crucial property within the Natural Language Processing community due to its effects on downstream model performance. |
| Approach: | They examine the level of human disagreement and the relative difficulty of each annotation task by training classifiers to perform the same task. |
| Outcome: | The authors show that human proficiency in identifying related intrinsic textual properties is low and that disagreements are high. |
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants (2022.naacl-main)
Copied to clipboard
| Challenge: | Dynamic Adversarial Data Collection (DADC) is a time-consuming and costly approach . DADC is based on training data collected from adversarial and out-of-domain settings . |
| Approach: | They propose a dynamic data collection approach that uses generator-in-the-loop models to provide real-time suggestions that annotators can approve, modify, or reject. |
| Outcome: | The proposed model is more robust in adversarial and out-of-domain settings and harder for humans to fool. |
Tokenization and the Noiseless Channel (2023.acl-long)
Copied to clipboard
| Challenge: | Subword tokenization is a key part of most NLP pipelines, but little is known about why some combinations lead to improved downstream model performance. |
| Approach: | They propose that good tokenizers lead to efficient channel usage . they propose that an optimal encoding assigns extremely long codes to low-frequency subwords . |
| Outcome: | The proposed tokenizers have a very strong correlation with BLEU in machine translation . the proposed function can be used to improve model performance in the downstream task . |
Want To Reduce Labeling Cost? GPT-3 Can Help (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Data annotation is labor-intensive and time-consuming for many NLP tasks. |
| Approach: | They propose to use GPT-3 to train models which are deployed for inference . they propose to combine pseudo labels from GPT3 with human labels . |
| Outcome: | The proposed method can be generalizable to many practical applications. |
A Rigorous Evaluation of LLM Data Generation Strategies for Low-Resource Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly used to generate synthetic textual data for training smaller specialized models. |
| Approach: | They evaluate the performance of large language models and their generation strategies in 11 different languages using 3 NLP tasks and 4 open-source LLMs. |
| Outcome: | The proposed generation strategies and their combinations yield strong results across 11 languages, including several extremely low-resource ones. |
Leitner-Guided Memory Replay for Cross-lingual Continual Learning (2024.naacl-long)
Copied to clipboard
| Challenge: | Various continual learning approaches have proposed to mitigate catastrophic forgetting by restricting the data buffer or limiting the data size of a model. |
| Approach: | They propose to use a human-inspired spaced-repetition technique to prioritize examples for cross-lingual continual learning. |
| Outcome: | The proposed approach significantly and consistently decreases forgetting while maintaining accuracy across natural language understanding tasks, language orders, and languages. |
Fair NLP Models with Differentially Private Text Encoders (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Encoded text representations often capture sensitive attributes about individuals, raising privacy concerns and making models unfair to certain groups. |
| Approach: | They propose an approach that combines privacy and adversarial training to learn private representations which induces fairer models. |
| Outcome: | The proposed approach improves on four NLP datasets and shows that privacy and fairness can positively reinforce each other. |
A Prism Module for Semantic Disentanglement in Name Entity Recognition (P19-1)
Copied to clipboard
| Challenge: | Xu et al., 2015) proposed a noise reduction mechanism to disentangle semantics of words . hard and soft attention mechanisms are used to reduce noise in NLP tasks . |
| Approach: | They propose a prism module to disentangle semantic aspects of words and reduce noise . they propose combining prism modules with downstream models to improve model performance . |
| Outcome: | The proposed method significantly improves the performance of baselines on named entity recognition (NER) tasks. |
Neuro-Symbolic Sentiment Analysis with Dynamic Word Sense Disambiguation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Traditional neural network models represent word senses as vectors that are uninterpretable for humans. |
| Approach: | They propose a framework that incorporates word Sense Disambiguation (WSD) by identifying and paraphrasing ambiguous words to improve sentiment predictions. |
| Outcome: | The proposed framework improves sentiment analysis accuracy and interpretability on a downstream task without ground-truth word sense labels. |
Pre-trained Speech Processing Models Contain Human-Like Biases that Propagate to Speech Emotion Recognition (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work has established that a person’s demographics and speech style affect how well speech processing models perform for them. |
| Approach: | They propose a method to detect bias in pre-trained models by using word embedding association tests in natural language processing to quantify bias in models' representations of different concepts. |
| Outcome: | The proposed method detects bias in pre-trained models and can have real-world effects. |
Annotation Sensitivity: Training Data Collection Methods Affect Model Performance (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Using an annotation instrument, the design of the annotation instrument and the instructions given to annotators can impact training data. |
| Approach: | They investigate the impact of an annotation instrument on training data . they collect hate speech and offensive language annotations in a tweet corpus . |
| Outcome: | The proposed model performs better on holdout conditions than on the standard model. |
Attributes as Textual Genes: Leveraging LLMs as Genetic Algorithm Simulators for Conditional Synthetic Data Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Genetic Prompt combines genetic algorithms with Large Language Models to augment synthetic data generation. |
| Approach: | They propose a framework that combines genetic algorithms with LLMs to augment synthetic data generation. |
| Outcome: | The proposed framework outperforms state-of-the-art models and shows robust performance across generator models. |
CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution (2026.acl-long)
Copied to clipboard
Xiangxi Zheng, Kuang He, Jiayi Hu, Ping Yu, Rui Yan, Yuan Yao, Peng Hou, Anxiang Zeng, Alex Jinpeng Wang
| Challenge: | Existing approaches to chart-to-code generation are constrained by data-centric limitations . authors present a new framework that redesigns both training and alignment data . |
| Approach: | They propose a data-centric framework that redesigns both training and alignment data for chart-to-code generation. |
| Outcome: | The proposed framework outperforms open-source baselines and is competitive with GPT-5. |
Schroedinger’s Threshold: When the AUC Doesn’t Predict Accuracy (2024.lrec-main)
Copied to clipboard
| Challenge: | AUC is a useful tool for evaluating diverse models without calibration. |
| Approach: | They show that the AUC yields an academic notion of accuracy that can misalign with actual accuracy observed in application. |
| Outcome: | The AUC yields an academic and optimistic notion of accuracy that can misalign with actual accuracy observed in application. |
Two Counterexamples to Tokenization and the Noiseless Channel (2024.lrec-main)
Copied to clipboard
| Challenge: | Nevertheless, Rényi efficiency is not perfect and the metric is difficult to evaluate because training multiple tokenizers can be prohibitively expensive and takes days or weeks. |
| Approach: | They propose to use Rényi efficiency as an intrinsic mechanism to evaluate a tokenizer for NLP tasks without the expensive step of training multiple models with different tokenizers. |
| Outcome: | The proposed metric is better correlated to downstream model performance than a percentile frequency metric. |